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Abstract 



The minimum value function appearing in Tikhonov regularization technique is very 
useful in determining the regularization parameter, both theoretically and numerically. In 
this paper, we discuss the properties of the minimum value function. We also propose 
an efficient method to determine the regularization parameter. A new criterion for the 
determination of the regularization parameter is also discussed. 

1 Introduction 



Let consider the generic model of inverse problems Kx = y where x € X and y G Y refer to 
the unknown and the data in a Banach space X and Y, and K is a nonlinear operator. The 



model can represent a variety of inverse problems arising in industrial applications including 
computerized tomography, inverse scattering and image processing. 
^ ■ Due to the ill-poisedness of the problem, a regularization method must be applied in order 

to retrieve x from the noisy data y and there are numerous works devoted for regularization 
methods. One of the most appealing regularization techniques is Tikhonov regularization, which 
has been studied from both theoretical and computational aspects by many authors. 

The Tikhonov regularization takes the minimizer of x a to the functional J a in an admissible 
set Q ad C X: 

x a := arg inf J a (x), (1) 

ZGQad 

where 

J a (x) = ip(x, y 5 ) + aip(x). 

Here a > is the regularization parameter compromising the data fitting and fidelity term 
(p and a priori information encoded in the restoration energy functional ij) of x. Commonly 
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used data fidelity functionals <p include \\Kx — y a ||?2, \\Kx — U^Wl 1 an d / {Kx — y s log Kx) and 
regularization functionals ip include (Bx, x)x with a bounded linear self-adjoint nonnegative 
operator B, \\x\\tv, etc. The set Q a d which describes a priori information for the solution xq is 
usually set to be weakly closed convex set in X. For instance, Q a d = Z n {x G Z \ \\x\\z < M} 
where Z <Z X is the another Banach space whose topology is stronger than that of X and M > 
is a given constant. For the detail of Tikhonov regularization, we refer to Baumeister [2], Engl, 
Hanke and Neubauer [3], Groetsch [6], Hofmann [9] and references therein. 

The selection of a parameter a is crucial for the stable solution and there is a significant 
amount of works for the development of methods for choosing a suitable parameter. Among 
others, we refer to [3HUE]. 

The minimum value function (see Definition [T]) appearing in Tikhonov regularization tech- 
nique is very useful in determining the regularization parameter. In [15 1 on several principles 
such as the generalized principle of discrepancy, the generalized principle of quasisolutions, the 
generalized principle of smoothing functional, and the principles are investigated by the calculus 
of the minimum value function. 

The value function calculus also gives an insight to well known conventional principles such as 
Morozov principle. For example, Ito and Kunisch proposed in |10] study the Morozov principle 
in terms of the minimum value function for nonlinear inverse problems. On the basis of the 
value function calculus, they propose a modified Morozov principle. As we see in section 4, 
other conventional principles can also be formulated in terms of the value function. 

In all the principles we have mentioned, each of the regularization parameters is determined 
with an equation including the value function and its higher derivatives. It can be computation- 
ally expensive to solve the equation numerically. In order to reduce the computational effort, 
an efficient method and an algorithm should be developed. 

A model function approach is proposed in [10J, in which an approximation (a model function) 
to the minimum value function is constructed and use the model function for the value function 
in their principle. Other principles and numerical algorithms can be found in Kunisch and Zou 
[T2], Xie and Zou [U]. 

In this paper, new results on the properties of the minimum value function are derived in a 
general set-up for problem (JTJ) , and a new principle for a choice of a regularization parameter is 
proposed using the minimum value function, which strongly relates to the principle of Reginska 
|14j also known as the minimum product criterion. We also propose a model function for the 
value function and employ the model function approach in several conventional principles to 
numerically compute the regularization parameters accurately and efficiently. 

The paper is organized as follows. In section 2, we give properties of the minimum value 



function. A model function is proposed in section 3 and the efficiency of the model function is 
verified in section 4. In section 5, a new principle for the regularization parameter is given. 



2 The minimum value function and its properties 

In this section, we investigate the properties of the minimum value function in a general frame- 
work : Consider the generic model of inverse problems Kx = y where K is a nonlinear operator 
from a Banach space X and another Banach space Y. We retrieve x from the noisy data y s 
using Tikhonov regularization technique, i.e., x is approximately obtained with the minimizer 

x a := arg inf J a (x). (2) 
Definition 1. The minimal value function F(a) of J a is the function of the parameter a defined 



as 



F(a) = inf J a {x). (3) 
Lemma 1. The value minimum function F(a) is (i) monotonically increasing and (ii) concave. 
Proof, (i): Let < a < a be given. For any x £ Qa,d, 

F(6l) < J&(x) = if(x,y s ) + atp(x) < ip(x,y s ) + aip(x). 
Taking the infimum with respect to x yields F(a) < F(a). 

(ii): Let ot\ > and a?2 > be given. Set 7 = (1 — t)ot\ + i«2 for < t < 1, then 



F((l -t)a 1 +ta 2 ) = inf JJx) = inf (cp(x,y d ) + ((1 - t)ai + ta 2 )ip(x) 

> (1 - t) w£(<p(x, y s ) + a x ^[x)) + t in£(<p(x, y 5 ) + a 2 ^{x)) 

X X 

= (l-t)F(ai) + tF(a 2 ). 
Hence F(a) is concave. □ 

Since F(a) is concave, it is continuous. 
Corollary 1. F(a) is continuous everywhere. 

Remark 1. LemmaUl does not require the existence of x 6 Q a d that achieves the infimum of 
J a . 

We examine the minimum value function more closely. Let D + F and D~F are one-sided 
derivatives of the value function F, i.e., 

1 . v , F(a) - F(a-h) _ „, , , F(a + h)-F(a) 

D + F(a) = lim — — -i '-, D~F(a = lim — i f y —?-. 

hlO h hio h 



Note that both limits exist for all a > and that D ± F(a) > 0. Indeed, for given a > 0, 

hi 

1 12 



let < hi < h 2 < a. Select t as t = 1 — r 1 < 1, then a — /ti = ta + (1 - t)(a - h 2 ). By the 



convexity of F(a), 

F(a - h x ) > tF(a) + (1 - t)F{a - h 2 ) = ( 1 - ^ ) F(a) + ^F(a - fr 2 ). 

\ h 2 J h 2 



Then 



F(c) - F(a - h 2 ) F(a) - F(« - hl )\ _ fa + fa 



h 2 hi J hi hi 



^H) F(a) - F(a - fe) + £{H 

™ , , , , F(a)-F(a-/t) 

Therefore, the limit hm > exists. 

HO h 

Here we list two the basic properties of D^F(a). 

1. Monotonicity: < D~F(a) < D+F(a) < D~F(a) < D+F(a), for all < a < a, 

2. Left and Right continuity: D + F(a) = lim D + F(a-h), and D~F(a) = lim D~F{a+h) 
for all a > 0. 

The differentiability of F at a guarantees the continuity of D^F at this point. Indeed, the mono- 
tonicity of D^F and the left continuity of D~F yield the inequalities D~F(a) = lim D~F(a + 

< lim D + F(a + h) < D + F(a). Now suppose F is differentiable at a, i.e, D+F(a) = D~F(a). 
H® 

Then from the inequalities it follows that lim D F{a + h) = D + F(a), which shows the continu- 
ity of D + F at a. Similarly it follows that D~F is continuous at a. The fact is used in section 



For the analysis of the minimum value function, we introduce the following function of 7 > 0: 

G( 7 ) := 7 ^(-)=7 mf Mx,/) + V)} 
7 zeQad 7 

= inf {ip(x) +<yip(x,y s )}. 

Note that once a result on F(a) is proved then the same result is true for G( 7 ) as well. For 
example, we can show that G( 7 ) is concave in exactly the same way as Lemma HJ Every results 
on Cr( 7 ) can be written in terms of F(a) by using the relation: 

D ± G(j)\ 7=a -i = F(a) - aD T F(a), (4) 

which can be verified without difficulty 



2.1 Asymptotic property of F(a) 
We study the asymptotic behavior of F(a). 



Proposition 1. 



limF(a) = inf <p(x,y 5 ), limaD ± F(a) = 0. (5) 

aj.0 zeQ ad aj.0 



Proof. Let C := inf xe Q ad </?(x, y 5 ). For any e > there exists x e € Q a d such that C < (f(x £ , y s ) < 
C + e. Then 

C < inf ip(x,y s )+a inf ^(jr) < F(a) < J a {x £ ) 
= tp(x £ ,y s ) + ail){x £ ) < C + s + aip(x £ ). 

By passing to the limit a J. and taking into account that e is arbitrary, we obtain C = limi^a). 

F(a) _ p( a _ M 

On the other hand, as we see above we have D + F{a) < for all h < a, here 

a is arbitrary fixed. Since limi ? (/3 — h) = C, we obtain 

h]a 

^ F ( t ,)<i.,/'°»-f l °- ft '. F M- c 

which means C < F(a) — aD + F(a). Therefore C < F{a) — aD + F(a) < F(a). From this 
inequality we get lim aD + F(a) = 0. 

Let {ctj} be a sequence that converges to as j — > oo. We pick ay such that &j < ay < 2ay 
to get the sequence {ay} such that ay — > 0. By monotonicity of D ± F(a) we have 

< a j D-F{a j ) < ajD + F(aj) 

= ajD + F(a 3 ) + (a, - dy)L>+F(dy) < 2a 3 D + F(a j ). 

Passing to the limit j — > oo yields lim aD~F(a) = 0. □ 

crj.0 

From this proposition it follows that F(a) is right continuous at 0. 
Proposition 2. 

lim = lim D ± F(a) = inf Mx). (6) 

Proof. Applying Proposition Q] to G{^) yields 

limG(7) = inf ij){x), lim 7 L> ± G(7) = 0, 

710 ^eQ a d 7|0 



or equivalently, 



lim G(a _1 ) = inf t/)(x), lim a- 1 D ± G(a~ 1 ) = 0. 

a^oo zeQad a~^oo 



By using the relation ([3]), we arrive at the desired results. □ 



2.2 Differentiability of F 

We investigate the relation between the derivative F' and the two terms, the fidelity functional 
tjj and the regularization functional ip. We also show a sufficient condition for the existence of 
F' in terms of ip and ip. 

Definition 2. We denote the set of solutions of the minimization problem inf J a {x) by A4 a . 

For simplicity of our argument we always assume the existence of the minimizer. 
It may happen that there exist two minimizers x a ^ x a € M a satisfying 

F(a) = <p(x a , y 5 ) + aip(x a ) = p(x a ,y 5 ) + aip(x a ) 

and 

cp(x a ,y s ) < if{x a ,y s ) and ip{x a ) > ip(x a ). 

That is, for a fixed a, the value <p(x a ,y s ) and ip(x a ) may vary depending on the choice of the 
minimizer x a S A4 a . Thus, it is possible that the maps a — > (p(x a ,y S ) and a — > ifj(x a ) will be 
multi-value functions. In what follows, we study the basic properties of those functions. We 
begin with the following inequality. 

Lemma 2. The following inequalities hold for all x a € M. a : 

D~F(a) < ip(x a ) < D + F(a), (7) 
F(a) -aD + F(a) < (p(x a ,y 5 ) < F(a) -aD~F(a). (8) 

Proof. For arbitrary a such that < a < a, 

F(a) = inf J & (x) < J&(x a ) = tp(x a ,y S ) + aijj(x a ). 

Thus, 

F(a) - F(a) > (p(x a ,y 5 ) + atp(x a ) - ip(x a ,y 5 ) - aip(x a ) 
= (a — a)il){x a ). 

Thus we obtain 

F(a) - F(a) , . , 
a — a 

By passing to the limit a | a, it follows that D + F{a) > ijj(x a ). Similarly, we obtain D~F(a) < 
ijj(x a ) and thus the inequality ([7]) is proven. 

We also obtain the second inequality ([8]) from ([7]) and the definition of F(a). □ 

Corollary 2. If F'(a) exists at a, then ip(x a ) and ip(x a ,y s ) are single valued at a and it holds 
that F'(a) = ip(x a ) and F(a) — aF'(a) = ip(x a ,y s ) for all x a G M. a - 



Note a monotone increasing (decreasing) function is differentiable except on a possibly count- 
able set. 

Corollary 3. There exists a possibly countable set N such that, 

• F is differentiable and the multi-value functions ip(x a ,y s ) and i/)(x a ) have single value on 
a G N C . 

• tp(x a ,y s ) = F(a) -aF'(a), ip(x a ) = F'(a) for all x a G M a if a G iV C . 

Corollary [3] guarantees the differentiability of F(a) except on a possibly countable set. 
Next we show the conditions for the differentiability of F(a) at all a > 0. Firstly, we define 
the ip boundness to state the assumptions. 

Definition 3. A sequence {x„}™ =1 C Q a d is i^-bounded if there exists constant M > such that 
sup n ip( x n) < M for all n G N. 

Assumption 1. • Let {x n } n C Q a d be a ^-bounded sequence. There exists a subsequence 
{x nk }k which weakly converges to an element x* G Q a d in the topology of X. 

• ip and tp are lower semi- continuous with respect to weakly convergence sequencers, i.e, if 
a subsequence {x n } n which weakly converges to an element x* G Q a d> then 

(p( x *) ^ liminf (p(x n ) and ip(x*) < liminf ip(x n ). 

n— >oo n— >oo 

Henceforth hereafter we assume the assumption holds. Then is guaranteed the existence of 
the solutions x^ in M. a such that tp(x^) = D ± F(a). 

Theorem 1. There exist x+ and x~ in A4 a such that ip(x^) = D + F(a) and ip(x~) = D~F(a) 
for all a > 0. 

Proof. Let a > fixed arbitrary and let h > be a parameter such that h « a and h — * 0. 
Let {x a _h}h be a minimizing sequence. Then from the monotonicity of F(a), 

Ja-h(xa-h) = F{a - h) < F(a) for all h > 0. 

F(a) 

Thus 4>(x a ^h) < r an d it follows that the sequence {x a -h}h is bounded. By Assumption 

a — h 

[H there exists subsequence of which we denote it by {x a _h}h, that converges weakly 

to an element x* G Q a d- Then by the continuity of F(a) and the lower semi-continuity of <~p and 
ip, it follows that 

F(a) = limF(a — h) > lim ip(x a _h, y s ) + a lim-i/'( 2; a-/i) 
h i° hlO hlO 

> <p{x*,y 5 ) + ahmilj(x a . h ) 

HO 

> ip(x*,y s ) + aiP(x*) = J a (x*)>F(a), 



and thus we have ip(x*) = lim tj){x a -h) an d x* G M. a . In what follows, we show that ip(x*) = 

HO 

D + F(a). 

By Lemma [21 

limL>-F(a - h) < hm^{ x a-h) < hmL> + F(a - h) = D + F(a). 
HO HO ho 

The last equality follows from the left continuity of D + F(a). Since lim _D~_F(g — h)= D + F(a), 

HO 

we obtain lim^(x a _/ l ) = D + F(a) and therefore = D + F(a). 

HO 

Similarly we can show the existence of the minimizer x~ G A4 a that satisfies ip(x~) = 
D~F(a). We complete the proof. □ 

Corollary 4. There exist elements x„ and x~ such that tp( x a) = max ^{ x ) and tp(x~) = 

xeM a 

min ip( x )- 
xeM a 

Corollary 5. If ip(x a ) = il){x a ) for all x a ,x a G M. a for all a > 0, then F'(a) exists and it is 
continuous for all a > 0. 

Corollary 6. Assume that the solution of minimization problem fTJj is unique for all a > 0, 
then F'(a) exists and it is continuous for all a > 0. 

The other properties for F(a) such as second differentiability is studied in |10j . 

As shown in Corollaries [5] and [6l both of the value F(a) and F^ (a) is obtained with the 

knowledge of x a , and the computation of —x a is not required. Moreover, we obtain F^ 2k \a) 

da 

d k 

and F 2k+1 (a) from ~^~k x ct f° r k > 1 provided that K is linear from Hilbert space X to another 
Hilbert space Y, tp(x,y 5 ) = \\Kx — y s \\y and ifj(x) = (Bx,x)x with a symmetric operator B 
such that dl]) has a unique solution, and X = Q a d, i-e, no constraint is imposed. 

Theorem 2. The function F{a) is infinitely differentiable at every a > 0. The derivatives 
F^ 2k \a) and F^ 2k+l \a) for each k > 1 are give with the k-th derivative OCq as 

F^ 2k \a) = C k ((K*K + aB)x < a k \x^) x 
F ( 2k +V (a) = -C k (1 + 2k) (Bx™ , 4*) ) x , 

2(2k + 1) 

where the constants C k are recursively defined as C k+ \ = C k with C\ = —2. 

k ~\~ 1 

The proof is based on the following lemma in |12j . 
Lemma 3 (|12j). The function x a is infinitely differentiable at every a > and its derivative 

(k) 

x a , for each k > 1, is the unique solution to the following equation: 

(K*K + aB)x {k) = -kBxg-V 



Proof of Theorem^ From Lemma[3]with k = 1, one obtains = 2{Bx ( a\x a ) x = -2((K*K+ 

aB)Xa\xa ) )x- 

Suppose F( 2k \a) = C k ((K*K + aB)x ( a ] ,x$)x with a constant C k , then Lemma [3] yields 



2C k (- ( k + 1) Bx™ , x (*) ) x + Cfc (flx W ,x*£')x 



F^ k+l \a) = 2C k ((K*K + aB)xi k+1 \x^) x + C k (Bx ( a k \x^) x 
= -C fc (l + 2fc)( J Bx^,xW)x. 

Then we have 

F (2k+2) {a) = _ 2Ck (l + 2k){Bx ( £ +1 \x^) x 

= 2C f + \ 2k \ x^\-(k + l)Bx^) x 
= C fc+1 (4 fc+1 ), (K*K + aB)x^) x . 

where C k+ \ is defined as C k+ \ = — ^7 — — -. By induction the assertion is valid. □ 

k + 1 

3 Pade approximations as model functions for linear inverse 
problems. 

In this section, we propose a model function for the value function for linear inverse problems. 
Firstly, we give a motivation for using the model function approach. 

A principle for determining a regularization parameter often requires solving an equation, 
for example, Morozov discrepancy principle takes the parameter a* that satisfies the equation 
\\Kx — y s \\x = 8 with noisy data y s of noise level 5. One can apply a Newton type iteration to 
solve the equation, however, the iteration could be numerically expensive. One strategy to reduce 
the computational effort is that: First, represent the equation in terms of the value function as 
F(a) — aF'{a) = 0. Then, construct a model function m(a) to F(a) and find the parameter 
oi m that satisfies the equation m{oi m ) — a m m'(a m ) = 0. If the model function approximates to 
the value function F, the parameter a m thus determined will give a close approximation to a*. 

We assume that K is a linear operator form Hilbert space X and a Hilbert space Y, Q a d = X 
(i.e., no constraint imposed), ip(x,y s ) = \\Kx — y ||y and ip(x) = (Bx,x)x with a symmetric 
operator B such that ([1]) has a unique solution. The solution is written as 

x a = (K*K + aB)- l K*y 5 . 

As we see in Theorem [21 the minimum value function F(a) is infinitely differentiable and thus 
it is reasonable to consider a rational function as a model function, which is briefly mentioned 



in [16]. We propose our model function to F(a) of the particular form 

s 2 P{a) s ,. 2 a n +p„_ia n_1 H pia + po 

m(a) = y v-— - = \\y \\ Y ; • 

The derivation of our proposed model function to F(a) bases on the following discussion: Just 
for simplicity we assume that X = Y = R l and B = I, although our discussion is valid for the 
infinite dimension framework. The singular value decomposition of K yields 

K = UZV T , £ = diag(ffi,...,o*), 

where o~\ > • • • > a T > o~ r+ i = ••• = &£ = Q (r = rank(i^)) are the singular values and 
U = [ui, ... ,ug\ and V = [t>i,... ,vi] are the orthogonal matrices, respectively. Then F(a) is 
represented as 

Ha) = £ ^-(nh 5 ? = \\y%, ~ t Jt-^IV 6 ) 2 - 0) 

Since we assume that A is highly ill-conditioned, the singular value a k decreases rapidly as k 
increases. As a result only the first few n-terms satisfying n « r in ([9]) will contribute to the 
sum. Thus we drop off the remaining terms and obtain the approximation 

F(a) « \\y% e (l - f f k „ 1 ) = \\y%t an+Pn - ian ~] + --- pia + P ° . (10) 

The Pade approximation to F(a) is constructed through the use of several minimizing ele- 
ments x a for different values of regularization parameter a. 

For a given interval /, let {a±, 02, a n } C / are n distinct parameters, which we call 
reference points in the following. We compute the function values and its derivatives at the 
reference points to determine 2n unknowns po, . . . ,p n -i, Qo, ■ ■ ■ Qn-i in m(a) by the linear system 



(a l ) = F(a i ), mW(a i ) = F<- 1 \a i ), i = l,2,...,n. (11) 



The more accurate model function will be obtained by imposing further conditions on the higher 
derivatives to m(a), i.e., we determine 4n unknowns po, . . . ,p2 n -i, Qo, ■ ■ ■ l2n-i by the system 

m(oci) = F{pu), m {l \ai) = F^fa), 
mW{a i ) = F®{a i ), m^( ai ) = F^( ai ), i = l,2,...,n. (12) 

For the solvability of the systems (jlip and (|12p one can refer to pQ. In the next section, we 
demonstrate that the model function approximates F(a) in the interval [ai,ct n ]. 

Remark 2. The differentiation of F^\a) yields F^(a) = 2{Bxa\x a ), and thus F^(a) = 
, Xa^). Therefore it seems that the computation of IS required for the 
evaluation of F^\a), however, from Theorem^ it is enough to solve the equation for x« for 
the evaluation of F^ (a) and is not necessary to compute Xq' . 



4 Numerical tests 



In this section, we present numerical tests to illustrate the efficiency of the proposed method 
using a linear inverse problems - "heat" which is generated from the Matlab package developed 
by Hansen [7]. <p and are of the forms f(x,y s ) = \\Kx — y^HjjLo with 50 x 50 matrix K and 
ijj{x) = ||a;||^5o. 

We use four conventional principles to determine the regularization parameters: the Moro- 
zov discrepancy principle, the damped Morozov principle, L-curve criterion and the minimum 
product criterion. 

The discrepancy principle ([13]) gives the regularization parameter ao as the solution of the 
equation 

<p(x a ,y 5 ) = 5 2 . (13) 

The damped discrepancy principle (|llj) is a modification of the discrepancy principle that 
consists in determining aap such that 

ip(x a ,y 5 ) + a^(x a ) = 5 2 , (14) 

In this test, we fix 7 = 1. 

L-curve criterion proposed in [8j gives the regularization parameter which attains the 
largest magnitude of curvature n{a) of the curve 

{(logi;(x a ),log(p{x a ,y 5 )) \ a > 0}. 

The curvature n(a) can be written as p2] 

K[a) = 3- \ip + ayj H . (15) 

The minimum product criterion [T3] takes the regularization parameter olmp as a local 
minimum point of the function 

%(a) := <p(x a ,y 6 )^(x a ), 7 > 0. (16) 

In this test, we fix 7 = 1. 

We rewrite all the principles in terms of F by using the equations 97(a) = F{a) — aF'(a), 
ip(a) = F'{a) and ip'(a) = F"{a) to employ model function approach. 

Our purpose here is to identify the parameters «m, odm, ol> «a/p numerically with high 
accuracy by employing model function approach. Note that we do not attempt to verify the 
effectiveness of those conventional principles for the inverse problem. We also note that our 



approach is not restricted to these principles. We can apply the approach to any principles that 
can be formulated in terms of the value function. Although the size of the matrixes K for the 
specific example is fixed to be 50 x 50, our approach is available for much larger scale inverse 
problems. 

The noisy data y s is generated by adding random noise to the exact data y so that 

,<5|| „ 



y - y 



|y||R5o 



G {0.01,0.03,0.05}. 



The regularization parameter a in Morozov principle is computed for each noise level as 
follows: we first compute (p(x a ,y s ) in for 100 uniformly distributed a-values in the interval 
[10~ 8 , 10 -0 ' 5 ] and find apprioximate optimals a and a that are very close, i.e, \a — a\ « 1 
and that satisfy ip(xa,y S ) < 5 2 < f(xa,y S )- Then a much smaller interval including these 
parameters is choosen to compute an accurate a M . The other parameters a DM ,a L ,a MP in 
different principles are computed in a similar manner. 

In each noise level, an approximation to the optimal parameter a M in Morozov principle 
is computed by employing model functions as follows: Firstly, we construct the model func- 



tion mi (a) of the form mi (a) 



^,|2 « 4 +P3® 3 +P2& 2 +Pice + po 
R5 ° a 4 + q 3 a 3 + q 2 a 2 + q±a + qo ' 



The eight unknowns 



{Pfc}fc=0' {%}fc=o m thi s model function are determined by solving the linear system (jlip where 
the value F(ai) and F^(ai) are obtained at four points a\ E {10 -8 , 10 -5 - 5 , 10~ 3 , 10~ a5 }. 
Then we solve the equation mi (a) — am'^a) = 5 2 , which is the model function version of 



(p(x a ,y 5 



S 2 . We denote the solution by af 1 . We also construct another model func- 



5|2 a 8 +p7<y. 7 -\ hpia + po 



whose sixteen unknowns {pk}1 = n, {%} 



lfc=0 



tion m a (a) = \\y \\ m 8 - 

a s + <? 7 a'H Vqia + qo 

are determined by solving the linear system (|11|) where up to third derivatives F(cti), F^(ai), 
FW(ati) and F( 3 )(a;) at the same points are used. We denote the solution of the equation 



7712 (a) 



am, a 



S z by a^f . The approximated parameters in the other principles are com- 



puted in similar manner using the model function mi (a) and 7712(a) and we denote them by 



a 



DM „,DM 



ft 



(damped Morozov for A = 1), af , a\ (L-curve) and a^ p , a^ p (minimum product 



for 7 = 1), respectively. All the computed parameters a 



M 



a 



M 
1 j 



a¥ etc in each noise level 



are reported in from Table [T] to Table [U The authors observed that the first derivative m'i(a) 



Table 1: a M , af and af. Morozov. 



e 


0.01 




0.03 


0.05 


£ 


„ M 

a 


3.81 x 10" 


-5 


1.39 x 10" 4 


2.68 x 10" 4 


„DM 

a 


„M 

Qi 


4.22 x 10" 


■5 


5.97 x 10~ 5 


7.22 x 10~ 5 


DM 

ax 


„M 

a 2 


3.87 x 10" 


-5 


1.25 x 10~ 4 


2.55 x 10~ 4 


DM 
«2 



Table 2: a DM , a? M and a? M . d-Morozov. 



0.01 0.03 0.05 

1.85 x 10~ 6 1.17 x 10" 5 2.84 x 10~ 5 

1.81 x 10 -6 1.00 x 10~ 5 1.84 x 10~ 5 

1.85 x 10 -6 1.17 x 10~ 5 2.74 x 10~ 5 



Table 3: a L , a\ and a% . L-curve. 

e 0.01 0.03 0.05 

a L 2.80 x 10~ 6 2.84 x 10" 5 7.22 x 10 -5 

ai 3.05 x 10~ 6 5.69 x 10" 6 3.86 x 10 -4 

a£ 2.52 x 10 -6 2.01 x 10~ 5 9.36 x 10~ 5 



Table 4: a MP , af p and af p . Minimum product. 

e 0.01 0.03 0.05 

a MP 8.50 x 10" 7 1.90 x 10" 5 7.22 x 10 -5 

af p 1.33 x 10" 6 9.72 x 10" 6 1.39 x 10~ 4 

ai IP 8.79 x 10~ 7 1.81 x 10 -5 7.47 x 10 -5 



did not approximate to F'(a). On the other hand m' 2 (a) was observed to give a very good 
approximation to F\a). 

The equation (114j) is written in terms of F as F(a) = 5 2 . This means that the parameter 
a DM is determined using only F. Thus it is enough to give a good approximation to F to 
compute an approximation to a DM . As we expect, the parameters af M and a 2 M in damped 
Morozov principle are very good approximations to a DM . 

The parameters , a\ and a^ p for all nose level are not so accurate. This is because the 
equations (|13fl . (|15fl . (|16p contain the first derivative of F and m^(a) does not approximate to 
F'{a). 

To give a better approximation to a M , a L and a Mf> , the model function must approximate 
to i 7 " in high accuracy and our model function 7712(a) will be the candidate. Table [T] and [3] 
show that the parameters determined using 7712(0;) are very accurate. On the other hand, a 2 
are not so accurate, although they are acceptably close to or. An accurate second derivative of 
F is also required for determining the parameter a L . Figure Q] and [2] show the curvature of the 
L-curve n(a) with its numerical approximations K appro (a) obtained by using mi (a) and 7712(a) 
respectively when the error e = 0.05. The four reference points a; G {10~ 8 , 10" 5 5 , 10" 3 , 10" 5 } 
used to construct mi (a) and 1712(a) are depicted by bullets (•) on the curve n(a) in each Figure. 
The approximation K appro (a) ( dashed line ) in Figure Q] completely fails to approximate 



L-curve curvature L-curve curvature L-curve curvature 




Figure 1: n(a) (solid line -) and Figure 2: n(a) (solid line -) and Figure 3: «(a) (solid line — ) and 

Kap P ro(a) (dashed line ) obtained K ap pro(a) (dashed line ) obtained n apP ro(a) (dashed line ) obtained 

by using mi (a) with four reference by using 7712(a) with four reference by using 7713(a) with five reference 
points ai (•). points ai (•). points an (•). 



to k(o). We used F(a.i) and F <yl \ai) for the construction of the model function m\{pt), and 

thus the second derivative m^\a) can not approximate to F^ 2 \a) which is contained in n(a). 

On the other hand, our 1712(01) gives better approximation to k as shown in Figure [21 although 

^appro(ct) does not much perfectly with k. To give more accurate K appro (a), we construct another 

model function of the form m 3 (a) = \\y S \\™ 5 o in , — 9 , ; — - — ; — - using F(a±i), ( OLi ) , 

a w + <7 9 a y H h q%a + q 

FW(ai) and F^^cn) at five reference points a { e {10~ 8 , 1(T 6 - 125 , 10" 4 - 25 , lO" 2 375 , lO" 05 }. 

Figure[3]depicts the curvature of the L-curve n(a) with its numerical approximations K appro (a) 
obtained by using 7713(0). The five reference points are also shown in the Figure. We observe 
that the model function 1713(a) yields the sufficiently good approximation K appro (a) that almost 
perfectly matches with the exact curvature n(a). This observation suggests that we should use 
more reference points to construct model function when we employ a principle that contains 
second derivatives of the value function. 

Remark 3. In a practical situation, an interval where a regularization parameter to be found is 
often much smaller than the interval (10 -8 , 10~ ' 5 ) used for our numerical test. If it is the case 
it is enough to construct a model function in a smaller interval. The number of reference points 
to be used for the construction of a model function can be reduced to two or three. 



5 A new criterion of the choice of the regularization parameter 

We propose a new criterion for the regularization parameter. Let us introduce a function T~(a) 
defined as 



r 7 («) = . ..m-pv . ( 17 ) 



7 T F 1+ T(«) 

(1 + 7) 1+ t a ' 
where 7 is a positive constant. 

dTy(a) 



Our new criterion takes the parameter as a local minimum a of Tj(a). Since 



da 



7 7 

F 7 (a)((l + 7)aF'(a) — F(a))a , and F(a) > for all a > 0, a 7 solves the equation 



(l + 7 )i+7 
(1 +<y)aF'(a) -F(a) = 0. 

The criterion is similar to the minimum product criterion by Regihska |14j . First, we note 

that the energy function ^j(a) in (|16p is written in terms of F(a) 

%(a) = (F(a) - aF'(a)fF'(a). (18) 

Suppose that F"(a) exists and F"(a) < 0. (A sufficient condition for the existence of the second 

derivative and the negativity can be found in [TO].) Since — f 7 (a) = (F(a)—aF'(a)) 1 ' l F"(a)(F(a) 

da 

(1 + j)aF' (a)) = (^(xq,, y 5 ) 7-1 F h '(a)(F(a) — (1 + j)aF' (a)), the regularization parameter de- 
termined by the criterion solves the equation (1 + j)aF'(a) — F(a) = 0. 

The relationship between (fTTJ) and (fTHj) follows from the next Proposition. 



Proposition 3. Let 7 > be a positive number. 

%{a) < T 7 (a), for all a > 0. (19) 
The equality holds if and only if a solves the equation 93(a) — jaip(a) = 0. 

Proof. Consider the inequality ab < a p /p + b q /q with p _1 = 7(1 + 7) -1 and q^ 1 = (1 + 7)~ x . 

7 7 1 1 

Substituting ip^+ia 2 (!+t) with a and ('yip) 1+7 a 2 ( 1+ T> with 6, it follows that 

V ji+7( 7 ^)i+7a 2 ( 1 +^ < — — - — = — ' V" 

1 + 7 a? 1 + 7 a 2 

1-7 

and the inequality holds if and only if a p = If, namely, ip(a) — ^/aip(a) = 0. Multiplying a 2 < 1+ t) 
and taking 1 + 7 power yields the desired inequality. □ 

Figure H] shows Ti(a) and ^i(a) in the interval (10 -10 , 10 -1 ) for certain linear inverse prob- 
lem. There exists a local minimum point around a = 1CP 5 where T\(a) and \?i(a) take the 



X) 




Figure 4: The graph of T±(a) (solid line — ) and ^fi(a) (dashed line ) 



same value. 

The advantages of our criterion are (i) the shape of T 7 (a) is sharper than ^ 7 (q;) and thus 
it is easier to detect the minimum point, (ii) T 7 (a) contains F(a) only, does not require the 
function F'(a) which can be discontinuous due to the nonuniqueness of an inverse problem. 

The effect of the parameter 7 in T 7 to the quality of the solution x a should be studied. We 
investigate both the applicability of the criterion to nonlinear problems and the effect of the 
parameter 7 in our future works. 



6 Conclusion 



We investigate the minimum value function for the Tikhonov regularization. We propose the 
model function for the minimum value function for linear inverse problems and verify its efficiency 



in the determination of the regularization parameter. We also propose a new criterion for the 
choice of the regularization parameter. Our criterion strongly relates to the minimum product 
criterion and is applicable to nonlinear inverse problems. 
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